Mathematical optimization schemes are the workhorse of machine learning - and at their core lies the derivative. Because of this an intuitive and rigorous understanding of the derivative - as well as other vital elements of calculus - serves one well in understanding mathematical optimization, and hence machine learning / deep learning more generally.

In this post we kick off our discussion of derivatives by exploring this idea in pictures before jumping into the math in future posts.

In the previous post we described in words and pictures what the derivative at a point is - in this post we get more formal and describe these ideas mathematically.